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In this paper we describe the development of the 
Mathematics Assessment Questionnaire, A -survey of t houghts 
and feelings (MAQ) , for students in grades 7-9, an assessment 
tool designed for use in JHS mathematics classrooms. We 
also describe the adaptation of the MAQ to an on-line 
computer-based administration (IBM compatible, 256k) . The 
computer adaptation of the MAQ includes the development of a 
teacher program disk to facilitate teacher access to student 
responses. Adapting to a computer-based administration 
supports: 1. flexible use by teachers of the MAQ for 
different groups of students; 2. teacher use of information 
for individuals or a class; 3. "anchoring" interpretation in 
the context of the statements to which students respond; 
and, through a Help feature, ready access to definitions and 
instructional strategy suggestions. 

Development of the Mathematics Assessment Ouostionnaire 

The MAQ is intended to assess selected aspects of JHS 
students' self -regulatory , affective, motivational and 
attributional beliefs about doing and learning about 
mathematical word problems in three classroom activity 
settings: 1. Purina class , when the teacher is working with 
the whole class; With others , when students are working in a 
small group with other students; and Homework , when students 
are working independently. The MAQ is intended to 
complement teacher assessment of student conceptual 
knowledge and procedural skills in mathematical problem 
solving. The MAQ focuses on awareness of thoughts and 
feelings when doing an individual non-routine mathematical 
word problem, and when learning about and doing problems in 
each of the three classroom activity settings. 



1 Revised version of a paper presented at the annual 
meeting of the American Educational Research Association, 
San Francisco, CA, April 1992. The research and development 
work reported here has been supported by the Ford Foundation 
and the Aaron Diamond Foundation. 
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The focus on classroom-level use of the MAQ led to 
several decisions. These decisions were: to use classroom 
activity settings as the context for assessing student 
perceptions, in order to link more closely with 
instructional activities; to use a more specific term, 
mathematical word problems, instead of "mathematics" in 
statements; and to use smaller sets of statements, since 
teachers can follow-up and check interpretations of student 
responses . 

The constructs represented in the MAQ computer version 
(MAQ-CV) are listed in Figure 1. The constructs are: 
metacoanition — awareness of monitoring, checking, strategy 
use (assessed following work on a non-routine word problem) , 
2 0 items; self-reaulatorv awareness in each of the three 
activity settings — 19 items During Class, 2 3 items Working 
with Others, and 9 items Homework; affective beliefs — 
confidence, anxiety, interest, value; motivational beliefs - 
~ internal learning goals, external performance goals; and 
attributional beliefs — internal stable controllable and 
unknown control. There are three-item clusters for each of 
the constructs in the affective, motivational and 
attributional beliefs within each activity setting (for a 
total of 24 items in each setting) . There are, then, a 
total of 143 statements in the MAQ-CV. (The rationale for 
selecting these constructs is described in Tittle & Hecht 
1990, and Hecht & Tittle, 1990; see Selected Related 
References attached.) 

Table 1 presents sample statements from the MAQ-CV, and 
shows the approach to writing statements- that was used for 
the MAQ. Typically, each statement includes a construct 
(e.g., anxiety), an activity setting (e.g.. During class), 
and a math term such as "math word problem." Brief 
definitions for each construct are given in Table 2. 

The developmental work that led to the MAQ-CV included: 
1. A feasibility study (1986-87) , in which related research 
literature was reviewed, sample statements were written, and 
experienced JHS mathematics teachers reviewed and evaluated 
the sample statements; 2. Small-scale studies (1987-88) of 
the meaningfulness of the metacognitive statements for 
students and teachers and of the type of problem best suited 
for these statements; 3. A pilot study, spring 1988, in 
which 3 00 sample items were administered (in three forms) in 
paper and pencil format to 1500 students in grades 7-9 in 14 
New York City public schools and one parochial school; 4. A 
large scale study, fall 1988, in which 162 items were 
administered in paper and pencil format to 1737 students in 
grades 7-9 in eight New York City public schools. 

During 1989-90 data analyses were carried out, 
including the development of a criterion-referenced (CRT- 
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type) scores — need and strength indicators — for the 
affective, motivational and attributional constructs. 
Summary indicators are provided for the three-item 
affective, motivational and attributional clusters of 
statements within each activity setting. The need indicator 
appears for a student when at least two of the three 
statements in a cluster are answered in a way to indicate 
need for teacher follow-up. For example, two of three 
anxiety statements answered TRUE or VERY TRUE. The strength 
indicators appears when all of the three statements are 
answered in a positive direction. Interpretations of the 
need and strength indicators are in Table 3 ; a sample 
calculation in Table 4. 

In 1990-91 and 1991-92 the MAQ was adapted to a 
computer-based administration (Student disk, the SMAQ) and 
the teacher program was developed (Teacher disk, the TMAQ) . 
As described earlier, there are currently 143 statements in 
the MAQ-CV. The student (SMAQ) and teacher (TMAQ) disks are 
now in a field trial (1992), with student and teacher 
evaluation data being collected. 

De monstration of the MAQ-CV; Student and Teacher 
Program Disks 

The main features of the SMAQ and TMAQ are listed in 
Figures 2 and 3. For the student disk, the program has the 
following features: 

. a separate survey disk is used for each student 
. a menu presents a choice of sections to start the 
survey 

. a summary keeps track of the number of statements 

completed for each section 
. a procedure is available to review/change responses 
. a NOTEBOOK feature is available to encourage students 

to write to their teachers at any place in the survey, 
. a NOTEBOOK record is kept of the MAQ location where 

the note is made 
. a Riddle appears when any MAQ section i.s completed 



For the teacher disk, the program has these features: 

. a data section, that assembles class results and 
. an edit feature for student data that 

identifies possible questionable response patterns 
. a utility feature that supports changing math problems 

in the metacognitive section 
. a menu-driven exploration of student and class 

responses including student notes in the NOTEBOOK 
. a HELP feature that includes definitions of terms, 

survey statements, questions to consider, and 

instructional strategies 
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Figures 4 and 5 present flow charts for the SMAQ and TMAQ, 
respectively, identifying the main features of each. Figure 
6 presents the flow chart for the TMAQ Help Screen. The 
TMAQ HELP menu is always available through a dedicated 
function key. 

As indicated in Figure 5, the user can decide to look 
at information for a class (CLASS), a group of students, or 
for individual students (STUDENT) - The information can 
always be examined statement -by-statement and the CRT-type 
need and strength indicators are also available for 
individual students and for a class. 

As Figure 6 shows, the options for the HELP feature 
include: a definition of the construct and the statements 
in the survey, and suggested instructional strategies. The 
suggestions were developed in collaboration with mathematics 
teachers and mathematics teacher educators. 

Discussion 

The use of technology for administering the MAQ has 
some benefits for students in encouraging a self -paced 
reading of the statements. The technology also supports 
writing about thoughts and feeling in learning mathematics, 
and communicating the writing to teachers. To date we have 
worked with a small set of teachers (5) who have used 
different ways to administer the MAQ: a class in a computer 
lab during math period; individual students working before 
or after school in a computer lab; a remedial math lab of 
ninth grade students reading the questions aloud and asking 
questions if a statement was not clear, including a 
bilingual student translating for others; and one-on-one 
with an individual student, with the student talking aloud 
about the statements as well as entering responses to 
statements. 

From the perspective of class-level assessment, any and 
all of these uses are appropriate. Teachers can identify 
particular statements and use them for class or group 
discussions. There are no "norms standardized procedures 
or right and wrong answers. The goal is for the computer 
technology to support teacher access to systematic 
assessment information, and to support teacher exploration 
of the student information. How teachers explore the 
information and the meaning and use they make of the 
information is a key question for us. 

We are now doing "talk-aloud" or think aloud procedures 
with teachers looking at the responses of students in their 
classes, as earlier we did a similar procedure for students 
responding to the MAQ. Table 5 provides a partial 
transcript of one student doing a "talk aloud" with the MAQ. 
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Table 5 also provides a partial transcript of a teacher 
looking at her own class data, including some of the 
interpretations and instructional suggestions she made. 

The transcripts suggest the research opportunity the 
technology-based assessment provides. Vie can work on 
understanding student thinking about learning in different 
mathematics classroom activity settings incorporating these 
other perspectives, of their feelings, motivations, and 
awareness of their self-regulatory thoughts. We can also 
work on understanding the meanings teachers construct from 
these responses, particularly as these meanings relate to 
classroom instructional planning. 

In summary, the use of technology for a classroom-level 
assessment tool has broadened the assessment process for 
students — adding direct communication through notes to 
teachers. The use of technology has greatly expanded 
teacher flexible access to information, and in fact 
encourages teachers to examine the informiition from 
different perspectives. Further, the technology has 
supported integrating instructional suggestions for access 
by teachers when looking at student responses. 

The limitations in the example of the MAQ are the 
psychometric theory with which we started and the indirect 
assessment of these important attitudinal and self- 
regulatory constructs. By this we mean that we do not have 
a psychometric theory that is integrated with a cognitive 
perspective and the assessment process is isolated from on- 
going instructional activities. However, we have started to 
examine, from one perspective, the process of looking at how 
students and teachers think and feel about some aspects of 
learning and doing mathematical word problems in JHS 
classrooms. The computer-based teacher program provides an 
unusual opportunity to study teachers' thoughts about 
student assessment information, in the context of their own 
students' responses, in an area that has received little 
classroom attention in assessment — the self -regulatory , 
affective and motivational beliefs of students. 
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Table 1 

Sample Statexnents from the Mathematics Assessment 
Questionnaire Illustrating Constructs and 
Classroom Activity Settings 



S elf regulation; During class 

When I can think of another way to solve a word problem, I 
volunteer to show the class. 

Self regulation; Working with others 

I say to the other students if I think something should be 
worked differently. 

Self regulation: Homework 

I decide when is the best time to do my math homework word 
problems. 

Anxiety: During Class 

I get scared when I have to work a word problem on the 
board. 

Anxiety: Working with others 

I dread the thought of trying to solve a math word problem 
with other students. 

Anxiety: Homework 

I feel relaxed when I am doing word problems at home. 



Internal learning goals: Homework 

I like to do hard homework math word problems because I 
learn more math by v/orking them. 
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Table 2 

Definitions of Beliefs in the MAQ 



Metacocrnition : self-awareness of what done and strategies used 
when working a specific non-routine word problem. 

Self -Regulation t awareness of salf-directed strategi e s to learn 
as related to working Tnathexaatical word problems in the 
classroom activity setting. 



V alue ; the conviction that learning about mathematical word 
problems is worthwhile , useful or important . 

Interest : the liking to work or learn about mathematical word 
problems . 

Confidence ; the belief in one's ability to do or learn 
mathematical word problems* 

Anxiety ; worry , uneasiness or fear about doing or learning 
mathematical word problems. 

Internal Learning Goals ; internal or intrinsic motivations for 
learning of mathematical word problems — one finds it 
personally challenging and valuable. 

External Perfozinance Goals ; external motivations for learning 

mathematical word problems — one is motivated by grades or 
teacher approval. 

I nternal Stable Controllable : an attribut ion of success (During 
Class, Homework) or failure (Working With Others) for doing 
or understanding mathematical word problems to a cause which 
is internal AND changeable — e.g., effort. 

Unknown Control ; perceived confusion or inability to make sense 
out of why one succeeds or fails at doing mathematical word 
problems . 
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Table 3 

General interpretation of MAQ NEED and STRENGTH Indicators 



Need Strength 
Indicators Indicators 



Affective Beliefs: 

Value low high 

Interest low high 

Confidence low high 

Anxiety high low 

Motivational Beliefs (to learn/do> : 

Internal Learning Goals low high 

External Learning Goals high low 

Attributional Beliefs (causes of success/failure) : 

Internal Stable Controllable low high 
(e.g. , effort) 

Unknown Control high * 
confused about causes 

* Strength indicators are not relevant for Unknown Control. 

Table 4 

An example of the calculation of a "Need Indicator": Anxiety-Homework 

(* is the student's response) 

VERY TRUE SORT OF NOT VERY NOT AT 
TRUE TRUE TRUE ALL TRUE 

17. I feel nervous when I * 
think about doing hard 
word problems for homework. 

23. 1 feel relaxed when I am * 
doing math word problems 
at home. 

31. Doing word problems for * 
homework does not make 
me nervous . 

The student reports feeling anxious on two statements, indicated by a 
response in one of the two extreme categories — (Item 17, a VERY TRUE 
response; Item 23, a NOT AT ALL TRUE response); On Item 31 the student 
does not report feeling anxious. 

NEED is indicated since the student responded to AT LEAST TWO of the 
statements in a way to suggest ANXIETY. 



NEED is indicated when a student responds to AT LEAST TWO of the three 
statements in a way to suggest the need for follow-up (e.g., ANXIETY). 
STRENGTH is indicated when a student responds to ALL THREE statements 
in a way to suggest a strength (e.g, a LACK OF ANXIETY). 
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Table 5 

Computer Version 
Mathematics Assessiaent Questionnaire ; 
Teacher and Student "Talk Alouds" 



Teacher 

"This is what I like best about this program. This is interesting, 
because this kid is uncomfortable during instruction, but he doesn't 
show up here (pointing to the homev^ork setting). He's very introverted; 
but at home I guess he can do math better than he can in school. (She 
reviews other students who have a need with regard to Anxiety in the 
During Class and Working With Others settings). If it^s really true 
that they're anxious during instruction, I'm not sure if it has 
something to do with being called on to go to the board, or putting them 
in situations with other students where they don't feel they can either 
handle it or perform well, and if that were the case, I would probably 
try to arrange a different situation for them. Instead of just randomly 
calling them to the board, I may go over and work with them a little bit 
before the lesson starts so that they feel they have a grasp on it, so 
then I can call them to participate, and go to the board and get some 
positive feedback. I may try then to put them with other students that 
maybe are also anxious that need more help, where they can take charge a 
little bit, or something. But what happens with some of these students 
is because they're anxious, they're very difficult, they have a tendency 
to be dismissed as troublemakers, because you don't have time for all of 
them. 

Student: Activity setting: DURING CLASS 

Statement: I am afraid when I have to ask my math teacher a question about 
a word problem during class. 

Student response: "afraid. Well, sometimes I'm like afraid because if 
everyone else seems like they have gotten it and like I'm totally lost 
it could be kind of embarrassing. That used to happen, but not always, 
so it is sort of true ... sometimes if I'm just like lost I don't want^ 
to raise my hand or anything because I am afraid I'll be branded. If it 
is important and I don't understand it, like it is the day before a 
test, then I will just raise my hand... so I guess it is sort of true." 

Activity setting: WORKING WITH OTHER STUDENTS 

Statement: I dread the thought of trying to solve a math word problem with 
other students. 

Student response: "not at all true, because you can learn more that way." 
Activity setting: HOMEWORK 

Statement: I feel nervous when I think about doing hard word problems for 
homework. 

Student response: "...I feel kind of nervous ... cause I want to show that at 
least I can do good effort." 
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Figure 1 



Domain Specifications for the Mathematics Assessment Questionnaire 
Two Facets: Psychological Construct and Setting 

CONSTRUCT 



Metacognitive : 

Solving a math problem 

.Planning, defining objectives^ 

•Monitoring progress 



.Checking & evaluating_ 
.Strategies employed 



METACOGNITIVE ITEMS 

NOT LINKED TO SPECIFIC SETTING 
LINKED TO SPECIFIC PROBLEM 



ACTIVITY SETTING 



CONSTRUCT 



Self -regulation 
.awareness of self -directed 
strategies to learn and work 
(varies depending upon setting) 



Affective Beliefs 
.Value, utility_ 
. Interest 



. Confidence^ 
.Anxiety 



Motivations 

.Internal Learning Goals_ 



.External Performance Goals 

Attributions 

.Internal Stable Controllable_ 
.Unknown Control 



During 
Class 



Working 
with 
Others 



Doing 
Homework 
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Figure 2 



Features of Student Disk: SMAQ 
Mathematics Assessment Questionnaire-CV 



o A separate disk is used for each student 
o A flexible and varied use questionnaire disk 
— Use on different days 

— Use with individual students or a class 

— Use one-on-one, with a think-aloud procedure 
o A menu presents a choice of sections to start 
o A summary keeps track of the statements 

completed for each section 
o A procedure is available to review/change responses 
o A NOTEBOOK feature is available to encourage 

students to write to teachers at any time 
o A NOTEBOOK record is kept of the MAQ location 

where the note is made 
o A riddle appears when any MAQ section is completed 



Figure 3 

Features of Teacher Program Disk: TMAQ 



o A program designed for easy access to student and 

class responses 
o A data section, that assembles class results and 
o A feature for editing student data that identifies 

possible questionable response patterns 
o A utility feature to support changing mathematics 

problems in the metacognitive section 
o A menu-driven exploration of student and class 

rt^sponses including student notes in the NOTEBOOK 
o A HE!LP feature that includes 

— definitions of terms 

— v^uestionnaire statements 

— cfuestions to consider and 

--instructional strategies to try 
o A print-screen feature 
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Figure 4 
Flow Cliax^t of SMAQ 
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Figure 5 
Flow Chart of TMAQ 
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Figure 6 

Flow Chart of TMAQ Help Screens - Fl KEY 
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